System for image dehazing by modifying lower bound of transmission rate and method therefor

ABSTRACT

Disclosed is a system for image dehazing by modifying a lower bound of a transmission rate and a method therefor, whereby a clear image is obtained by removing components, such as fog or haze, from an image having low image quality due to fog or haze, the system configured such that a lower bound of a transmission rate representing how many rate of haze is mixed is calculated for each pixel, and an initial transmission rate is obtained by an exponentiation operation with the lower bound of transmission rate. By transmission rate correction that reserves an edge showing a large change relative to the initial transmission rate and processes a smooth area showing a small change with a low-pass filter, a final transmission rate is obtained such that a clear image is obtained by removing haze components.

CROSS REFERENCE TO RELATED APPLICATION

The present application claims priority to Korean Patent Application No. 10-2016-0053355, filed Apr. 29, 2016, the entire contents of which is incorporated herein for all purposes by this reference.

STATEMENT REGARDING SPONSORED RESEARCH

This research was supported by the MSIP (Ministry of Science, ICT and Future Planning) of Korea, under the ITRC (Information Technology Research Center) support program (IITP-2016-R2718-16-0011) supervised by the IITP (Institute for Information & communications Technology Promotion)

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates generally to a system for image dehazing by modifying a lower bound of a transmission rate and a method therefor. More particularly, the present invention relates to a system for image dehazing by modifying a lower bound of a transmission rate and a method therefor, in which a clear image is obtained by removing components, such as fog or haze, from an image having low image quality due to fog or haze.

Description of the Related Art

Recently, an image monitoring system or an image black box for automobiles or the like is used for detection or prevention of accidents. Further, in the case of an advanced safety vehicle, study is being conducted to provide a lane deviation or car collision warning system by extracting lanes and nearby vehicle information from the image acquired by a video camera.

In order to improve the performance of a computer vision application system or image process, a clear input image is needed. Particularly, when detecting or estimating an object or using the edge information of the image, a clearer image will bring a better result.

However, in case of an image taken outdoors, the brightness and the color reflected from an object are affected by particles in the air. Thus, its original color and brightness contrast are not provided, especially, in case that there are large particles in the air such as haze and smoke.

As for the conventional method for improving an image that includes fog, etc., there is Korean Patent Publication No. 10-2010-0021952 (hereinafter, ‘prior art’) in addition to a plurality of applications published and disclosed.

The method according to the prior art comprises a step of receiving the first luminance image of an image including airlight, and producing a map of atmosphere airlight based on the ratio of the standard deviation and the average of the first luminance image; and a step of outputting the second luminance image from which the atmosphere airlight is removed by subtracting a map of atmosphere airlight produced from the first luminance image. However, in the prior art a fogged image process is processed by a block unit, and not by a pixel unit.

On the other hand, a variety of methods for estimating a clear image by removing fog from a fogged image have been proposed. In the early stage, a method using a number of images or additional information besides an image was proposed to restore a fogged image to a fog-eliminated image. As the method using a number of images, the method using polarization [1, 2] was proposed. This method acquires two or more images taken with respectively different polarization filters installed in precisely a same location, and the depth value is calculated by using a method for measuring the polarized amount, and the fog is removed by using it.

These methods provide a very good image; however, there is a limitation that respectively different polarization filters have to be used in the same location. In a method [3, 4, 5], when simply using a number of images without using a polarization filter, fog value and depth information are obtained from two images taken under different weather environments to remove the fog. Kopf, et al. [6] proposed a method for removing fog by using depth information of an image instead of using a number of images, and the fog was removed by obtaining depth or texture information by using GPS information embedded in a camera and assuming concentration (density) of fog value is depth information.

The methods for removing fog by using a number of images or using a single image with additional information have a defect that they cannot be adapted to an image taken by a dynamically-moving camera because they need to secure image data in various conditions. Thus recently, a method for removing fog by using a single image is being studied.

Tan [8] proposed a method for removing fog by increasing brightness contrast. That is, fog was removed by using the characteristic that a clear image without fog had higher edge strength than a fogged image and the fog value did not change rapidly. In this method, the brightness contrast is highly improved, thus the shape and structure of an image are advantageously revealed. However, excessive increase of contrast may cause saturation, and a halo effect may occur in the section where the depth information is largely different. Fattal [9] proposed a method for restoring a fog-eliminated image by measuring the image reflection ratio through assumption that a reflection ratio measured within a constant image area has always the same vector direction.

He et al. [10] used the characteristic that a clear image has a higher chroma of color than a fogged image, and proposed a method that removes fog by the observation result that a pixel with high color sharpness in a clear image without fog has a very low channel value of one of R, G, B values, thus a color image without fog has a pixel with very low channel value in a certain area. However, in case only a luminance image is used because RGB color is used in the conventional method using only a single image, fog-elimination performance is degraded, and when a large size of filter is used, a halo effect occurs and a large calculation amount is required to refine a transmission rate. Thus there is difficulty in real time processing.

Tarel et al. [11] proposed a fog elimination method using a median filter in order to improve the calculation speed. However, if using a large size median filter, there is a disadvantage that the calculation speed is decelerated and a halo effect can occur.

-   [1] Y. Y. Schechner, S. G. Narasimhan, and S. K. Nayar, “Instant     dehazing of images using polarization,” in Proc. CVPR, pp. 325-332,     Hawaii, USA, December 2001. -   [2] S. Shwartz, E. Namer, and Y. Y. Schechner, “Blind haze     separation,” in Proc. CVPR, pp. 1984-1991, New York, USA, October     2006. -   [3] S. G. Narasimhan and S. K. Nayar, “Chromatic framework for     vision in bad weather,” in Proc. CVPR, pp. 598-605, SC, USA, June     2000. -   [4] S. G. Narasimhan and S. K. Nayar, “Contrast restoration of     weather degraded images,” IEEE Trans. Pattern Anal. Mach. Intell.,     vol. 25, no. 6, pp. 713-724, June 2003. -   [5] S. K. Nayar and S. G. Narasimhan, “Vision in bad weather,” in     Proc. ICCV, pp. 820-827, Corfu, Greece, September 1999. -   [6] J. Kopf, B. Neubert, B. Chen, M. Cohen, D. Cohen-Or, O.     Deussen, M. Uyttendaele, and D. Lischinski, “Deep photo: Model-based     photograph enhancement and viewing,” ACM Trans. Graphics, vol. 27,     no. 5, pp. 1-10, December 2008. -   [7] S. G. Narasimhan and S. K. Nayar, “Interactive deweathering of     an image using physical models,” In Workshop on Color and     Photometric Methods in Computer Vision, OCT. 2003 -   [8] R. Tan, “Visibility in bad weather from a single image,” in Proc     CVPR, pp. 1-8, Alaska, USA, June 2008. -   [9] R. Fattal, “Single image dehazing,” ACM Trans. Graphics, vol.     27, no. 3, pp. 1-9, August 2008. -   [10] K. He, J. Sun, and X. Tang, “Single image haze removal using     dark channel prior,” in Proc. CVPR, pp. 1956-1963, Miami, USA, June     2009. -   [11] Tarel, Jean-Philippe; Hautiere, Nicolas; “Fast visibility     restoration from a single color or gray level image,” Computer     Vision, 2009 IEEE 12th International Conference on, pp. 2201-2208,     September 2009.

The foregoing is intended merely to aid in the understanding of the background of the present invention, and is not intended to mean that the present invention falls within the purview of the related art that is already known to those skilled in the art.

SUMMARY OF THE INVENTION

Accordingly, the present invention has been made keeping in mind the above problems occurring in the related art, and the present invention is intended to propose a system for image dehazing by modifying a lower bound of a transmission rate and a method therefor, in which real-time processing is enabled in an embedded processor, such as a mobile phone, by an amount of calculation smaller than that of a conventional method.

The present invention is further intended to propose a system for image dehazing by modifying a lower bound of a transmission rate and a method therefor, in which a clear image is provided by removing components that lower visibility from an image having lowered visibility due to mixture of light and color components of an object and light and color components, such as haze, fog, clouds, and the like, in the air.

The present invention is further intended to propose a system for image dehazing by modifying a lower bound of a transmission rate and a method therefor, in which it is possible to maintain dehazing performance without a halo effect, and an amount of calculation is reduced to enable real-time processing in an embedded processor, such as a mobile phone.

In order to achieve the above object, according to one aspect of the present invention, there is provided a system for image dehazing by modifying a lower bound of a transmission rate, the system including: an atmospheric brightness value calculation unit for calculating an atmospheric brightness value A^(c) corresponding to a distance farthest away from a camera relative to an input signal I^(c)(x); a transmission rate lower bound calculation unit for calculating a lower bound of a transmission rate t_(LB)(x) by receiving the input signal and the atmospheric brightness value from the atmospheric brightness value calculation unit; an exponentiation operation unit for operating an initial transmission rate [t_(LB)(x)]^(P) by multiplying the lower bound of transmission rate P times; a transmission rate correction unit for calculating a final transmission rate by correcting the initial transmission rate such that an output restored image is dehazed without a halo effect; a restored image calculation unit for calculating a dehazed image through operation; and a postprocessing unit for outputting a final restored image J^(c)(x) by increasing brightness contrast.

Preferably, the transmission rate lower bound calculation unit calculates the lower bound of transmission rate t_(LB)(x) by Equation 11.

$\begin{matrix} {{t_{LB}(x)} = {1 - {\min_{\; {c \in {\{{R,G,B}\}}}}\left( \frac{I^{c}(x)}{A^{c}} \right)}}} & \left\lbrack {{Equation}\mspace{14mu} 11} \right\rbrack \end{matrix}$

Further, preferably, the restored image calculation unit calculates the dehazed image by Equation 12.

$\begin{matrix} {{J^{c}(x)} = {\frac{{I^{c}(x)} - A^{c}}{\max \mspace{11mu} \left( {{t(x)},t_{0}} \right)} + A^{c}}} & \left\lbrack {{Equation}\mspace{14mu} 12} \right\rbrack \end{matrix}$

Further, preferably, the final transmission rate is obtained by applying an edge reserved low-pass filter to the initial transmission rate.

Further, preferably, the image dehazing system further includes: an initial transmission rate separation unit arranged to separate the initial transmission rate into low-frequency components t_(L)(x) and high-frequency components t_(H)(x); and a compression coefficient computing unit arranged such that the high-frequency components are low-pass filtered to obtain a compression coefficient value ω(x) proportional to an absolute value of the initial transmission rate having been low-pass filtered as shown in Equation 13, and the final transmission rate t(x) is obtained by adding a value t_(h)(x) obtained by multiplying the compression coefficient value by the high-frequency components and the low-frequency components of the initial transmission rate.

$\begin{matrix} \begin{matrix} {{t_{h}(x)} = {{w(x)} \times {t_{H}(x)}}} \\ {= {{\min \left( {\frac{{F\left( {t_{H}(x)} \right)}}{D},1} \right)} \times {t_{H}(x)}}} \end{matrix} & \left\lbrack {{Equation}\mspace{14mu} 13} \right\rbrack \end{matrix}$

Further, preferably, the image dehazing system further includes: an initial transmission rate separation unit arranged to separate the initial transmission rate into low-frequency components t_(L)(x) and high-frequency components t_(H)(x); a threshold value processing unit for expressing a position of a component of the high-frequency components t_(H)(x) of the initial transmission rate greater than a threshold value as 1; an independent component removal unit for converting independently existing 1 output from the threshold value processing unit into 0; an extension processing unit configured to perform an extension processing on a component expressed as 1 output from the independent component removal unit so as to output a value R(x) having a position of an extended area expressed as 1 and a position of a rest expressed as 0; a multiplication operator for obtaining modified high-frequency components t_(h)(x) by multiplying the value R(x) by the high-frequency components of the initial transmission rate; and an addition operator for outputting the final transmission rate t(x) by adding the modified high-frequency components t_(h)(x) of the initial transmission rate and the low-frequency components t_(L)(x).

Preferably, a method for image dehazing by modifying a lower bound of transmission rate, wherein the method uses a system for image dehazing by modifying a lower bound of transmission rate, the image dehazing system including an atmospheric brightness value calculation unit, a transmission rate lower bound calculation unit, an exponentiation operation unit, a transmission rate correction unit, a restored image calculation unit, and a postprocessing unit, the method includes: calculating a brightness value relative to an atmospheric brightness value of an input image by using the atmospheric brightness value calculation unit; calculating a lower bound of transmission rate for each pixel using information on the input image and the atmospheric brightness value by using the transmission rate lower bound calculation unit; obtaining an initial transmission rate by an exponentiation operation with the lower bound of transmission rate by using the exponentiation operation unit; correcting the input image to have a strong edge relative to the initial transmission rate by smoothing such that an output image restored by the transmission rate correction unit is provided to be a dehazed clear image without a halo effect; calculating a final transmission rate to be used to dehaze by using the restored image calculation unit; and outputting a dehazed restored image processed by image restoration and postprocessing by using the postprocessing unit.

According to the present invention as set forth, it is possible to provide a clear image by removing components that lower visibility, from an image having lowered visibility due to mixture of light and color components of an object and light and color components, such as haze, fog, clouds, and the like, in the air.

Further, according to the present invention, it is possible to maintain dehazing performance without a halo effect, and an amount of calculation is reduced to enable real-time processing in an embedded processor, such as a mobile phone.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects, features and other advantages of the present invention will be more clearly understood from the following detailed description when taken in conjunction with the accompanying drawings, in which:

FIG. 1 illustrates an overall diagram of an image dehazing system modifying a lower bound of a transmission rate according to the present invention;

FIGS. 2A through 2J illustrate a result of simulation according to the image dehazing system of FIG. 1;

FIG. 3 illustrates an overall block diagram of the system for image dehazing by modifying a lower bound of a transmission rate and a method therefor;

FIG. 4 illustrates an exemplary block diagram of transmission rate calculation method 1;

FIGS. 5A through 5D illustrate images dehazed by the transmission rate calculation method 1 of FIG. 4;

FIG. 6 illustrates an exemplary block diagram of transmission rate calculation method 2;

FIGS. 7A through 7H illustrate images dehazed by the transmission rate calculation method 2 of FIG. 6;

FIG. 8 illustrates an exemplary block diagram of transmission rate calculation method 3; and

FIGS. 9A through 9H illustrates image dehazed by the transmission rate calculation method 3 of FIG. 8.

DETAILED DESCRIPTION OF THE INVENTION

Specific features and advantages of the present invention will be more clearly understood from the following detailed description when taken in conjunction with the accompanying drawings. In case functions related to the present invention and specific description for the configuration unnecessarily obscure the gist of the present invention, it is noticed that the specific description will be omitted.

According to the present invention, an initial transmission rate is obtained by an exponentiation operation with a lower bound of transmission rate, and the initial transmission rate is corrected to obtain a transmission rate for a haze component, whereby haze removal is performed. In a conventional method, the initial transmission rate is obtained through area processing, such as filtering, wherein in the process of refining a transmission rate, a large amount of calculating is required to reduce a halo effect. On the contrary, in the present invention, the initial transmission rate is obtained through processing each pixel, and a small amount of calculating is required in the process of transmission rate correction.

Further, the present invention is configured such that when RGB color signals are input, operation is mainly performed with information on the transmission rate but not performed with each color channel, whereby it is possible to reduce the calculation amount. Further, it is possible to perform most operations, such as transmission rate lower bound calculation and exponentiation operation, at the same time using LUT (lookup table).

In terms of transmission rate correction requiring the largest amount of calculating, a moving average filter that requires a small amount of calculating is applied to improve performance. The present invention is advantageous in that an amount of calculating is reduced to enable real-time processing in an embedded processor, such as a mobile phone, and it is possible to realize excellent dehazing performance. The present invention is further advantageous in that since the present invention has excellent dehazing performance with brightness signals, it is possible to apply various color formats (YCb, Cr, YUV, and so on) used in multimedia systems.

Hereinbelow, the present invention will be described in detail with reference to the accompanying drawings.

First, FIG. 3 illustrates a diagram of an image dehazing system modifying a lower bound of a transmission rate according to an embodiment of the present invention. As shown in FIG. 3, the image dehazing system by modifying a lower bound of a transmission rate may include: an atmospheric brightness value calculation unit 300; a transmission rate lower bound calculation unit 301; an exponentiation operation unit 302; a transmission rate correction unit 303; a restored image calculation unit 304; and a postprocessing unit 305.

The atmospheric brightness value calculation unit 300 is provided for calculating an atmospheric brightness value A^(c) corresponding to a distance farthest away from a camera relative to an input signal I^(c)(x).

The atmospheric brightness value calculation process is a process of obtaining the atmospheric brightness value A^(c) corresponding to a distance farthest away from the camera. The atmospheric brightness value is obtained as follows. As shown in Equation 10, a minimum value of RGB color channel for each pixel is calculated, and the atmospheric brightness value A^(c) is obtained by making a mean value of RGB values of pixels, which have the minimum value obtained by Equation 10 larger than a predetermined threshold value, of pixels on a current image.

min I(x)=min_(cε{R,G,B})(I ^(c)(x))  [Equation 10]

The transmission rate lower bound calculation unit 301 is provided for calculating a lower bound of transmission rate t_(LB)(x), and performs Equation 11 by receiving the hazy input signal I^(c)(x) and the atmospheric brightness value A^(c) output from the atmospheric brightness value calculation unit. Herein, the transmission rate lower bound calculation is capable of calculating the lower bound of transmission rate t_(LB)(x) through operation of Equation 11.

$\begin{matrix} {{t_{LB}(x)} = {1 - {\min_{\; {c\; \in {\{{R,G,B}\}}}}\left( \frac{I^{c}(x)}{A^{c}} \right)}}} & \left\lbrack {{Equation}\mspace{14mu} 11} \right\rbrack \end{matrix}$

In [Equation 11], the hazy input signal I^(c)(x) and the atmospheric brightness value A^(c) output from the atmospheric brightness value calculation unit are input; Min_(c)ε_({R, G, B})(I^(c)(x)) is an operation for obtaining the minimum value of RGB color channel in an x coordinate; and A^(c) is an output from the atmospheric brightness value calculation unit.

The exponentiation operation unit 302 is provided for operating an initial transmission rate [t_(LB)(x)]^(P) by multiplying the lower bound of transmission rate P times. In the process of the exponentiation operation, [t_(LB)(x)]^(P) is output by multiplying the lower bound of transmission rate P times, wherein [t_(LB)(x)]^(P) corresponds to the initial transmission rate. The initial transmission rate, as an output through the exponentiation operation, includes information on haze components included in an input image and reflection components unique to an object, whereby in the case of dehazing using the initial transmission rate, a dehazing effect may be reduced since transmission rate for the reflection components is included.

The transmission rate correction unit 303 is provided for calculating a final transmission rate by correcting the initial transmission rate such that an output restored image is dehazed without a halo effect. In the transmission rate correction process is a process of extracting a transmission rate for haze components from the initial transmission rate. The haze components have a constant value locally. On the contrary, the reflection components vary according to changes in brightness (colors) included in the image. Accordingly, the transmission rate for the haze components included in the initial transmission rate corresponds to low-frequency components, and the transmission rate for the reflection components corresponds to high-frequency components.

Thus, the transmission rate for the haze components is obtained by low-pass filter processing. However, when the transmission rate obtained by low-pass filter processing is directly applied to dehazing, dehazing performance is good but a halo effect occurs near an edge where changes in brightness are large. Accordingly, in the transmission rate correction process, by smoothing the initial transmission rate while having a strong edge relative to the initial transmission rate, the final transmission rate t(x) is calculated.

The restored image calculation unit 304 is provided for calculating a dehazed image through operation. The restored image calculation unit 304 is configured such that the dehazed image is calculated by Equation 12.

The dehazed image may be output by performing Equation 12 with the input image I^(c)(x) and the final transmission rate t(x).

$\begin{matrix} {{J^{c}(x)} = {\frac{{I^{c}(x)} - A^{c}}{\max \mspace{11mu} \left( {{t(x)},t_{0}} \right)} + A^{c}}} & \left\lbrack {{Equation}\mspace{14mu} 12} \right\rbrack \end{matrix}$

In Equation 12, J^(c)(x) refers to a brightness value of output color channel c in the x coordinate; I^(c)(x) refers to a brightness value of input color channel c in the x coordinate; A^(c) refers to an output from the atmospheric brightness value calculation unit; t(x) refers to the transmission rate (as a value expressing a degree of haziness, 0 referring to opacity, and 1 referring to transparency) in the x coordinate; and to refers to a minimum transmission rate for preventing noise amplification caused when the transmission rate is very small.

The postprocessing unit 305 is provided for outputting a final restored image J^(c)(x) by increasing brightness contrast.

The image dehazing system according to the present invention may be configured such that the final transmission rate is obtained by applying an edge reserved low-pass filter to the initial transmission rate.

The image dehazing system may further include: an initial transmission rate separation unit arranged to separate the initial transmission rate into low-frequency components t_(L)(x) and high-frequency components t_(H)(x); and a compression coefficient computing unit arranged such that the high-frequency components are low-pass filtered to obtain a compression coefficient value ω(x) proportional to an absolute value of the initial transmission rate having been low-pass filtered as shown in Equation 13, and the final transmission rate t(x) is obtained by adding a value t_(h)(x) obtained by multiplying the compression coefficient value by the high-frequency components and the low-frequency components of the initial transmission rate.

$\begin{matrix} \begin{matrix} {{t_{h}(x)} = {{w(x)} \times {t_{H}(x)}}} \\ {= {{\min \left( {\frac{{F\left( {t_{H}(x)} \right)}}{D},1} \right)} \times {t_{H}(x)}}} \end{matrix} & \left\lbrack {{Equation}\mspace{14mu} 13} \right\rbrack \end{matrix}$

The image dehazing system may further include: an initial transmission rate separation unit arranged to separate the initial transmission rate into low-frequency components t_(L)(x) and high-frequency components t_(H)(x); a threshold value processing unit for expressing a position of a component of the high-frequency components t_(H)(x) of the initial transmission rate greater than a threshold value as 1; an independent component removal unit for converting independently existing 1 output from the threshold value processing unit into 0; an extension processing unit configured to perform an extension processing on a component expressed as 1 output from the independent component removal unit so as to output a value R(x) having a position of an extended area expressed as 1 and a position of a rest expressed as 0; a multiplication operator for obtaining modified high-frequency components t_(h)(x) by multiplying the value R(x) by the high-frequency components of the initial transmission rate; and an addition operator for outputting the final transmission rate t(x) by adding the modified high-frequency components t_(h)(x) of the initial transmission rate and the low-frequency components t_(L)(x).

Meanwhile, a method for image dehazing by modifying a lower bound of transmission rate, wherein the method uses a system for image dehazing by modifying a lower bound of transmission rate, the image dehazing system including: an atmospheric brightness value calculation unit, a transmission rate lower bound calculation unit, an exponentiation operation unit, a transmission rate correction unit, a restored image calculation unit, and a postprocessing unit, the method may include: calculating a brightness value relative to an atmospheric brightness value of an input image by using the atmospheric brightness value calculation unit; calculating a lower bound of transmission rate for each pixel using information on the input image and the atmospheric brightness value by using the transmission rate lower bound calculation unit; obtaining an initial transmission rate by an exponentiation operation with the lower bound of transmission rate by using the exponentiation operation unit; correcting the input image to have a strong edge relative to the initial transmission rate by smoothing such that an output image restored by the transmission rate correction unit is provided to be a dehazed clear image without a halo effect; calculating a final transmission rate to be used to dehaze by using the restored image calculation unit; and outputting a dehazed restored image processed by image restoration and postprocessing by using the postprocessing unit.

Hereinbelow, reference will be made in greater detail to the present invention.

In methods for dehazing using one image, the following modeling equation is commonly used.

I(x)=J(x)t(x)+A(1−t(x))  [Equation 1]

Herein, I(x) is a value of the xth pixel of the hazy image obtained by a camera; J(x) is a dehazed clear image; and A is the atmospheric brightness value of a pixel in the image, which is the farthest away from the camera. t(x) is the transmission rate, and in general, the transmission rate t(x) decreases exponentially according to a distance, as the following Equation 2.

t(x)=e ^(−βd(x))  [Equation2]

Herein, β is scattering coefficient of air, and d(x) is a distance between the camera and a point in a space corresponding to the xth pixel. A value of the scattering coefficient β is related to s particle size in the air, wherein β approaches 1 for big particles, such as rain drops and heavy haze particles, while it approaches 0 for small particles when the weather is clear. Accordingly, in the case where the scattering coefficient β is constant, the transmission rate for a distant location, such as the sky, approaches 0, and accordingly, I(x)≅A in [Equation 1], and I(x)≅J(x) since a pixel in very close location has transmission rate approaching 1. Accordingly, a bright pixel in the image can be assumed as a case of a far location and heavy haze, and the transmission rate t(x) has a small value.

In order to remove haze, A and t(x) are obtained from the input image I(x) obtained by the camera, and the final dehazed image J(x) is restored by using A and t(x). The transmission rate can be obtained from Equation 1 and restoration value can be obtained from Equation 3 and Equation 4.

$\begin{matrix} {{t(x)} = \frac{{I(x)} - A}{{J(x)} - A}} & \left\lbrack {{Equation}\mspace{14mu} 3} \right\rbrack \\ {{J(x)} = {\frac{{I(x)} - A}{t(x)} + A}} & \left\lbrack {{Equation}\mspace{14mu} 4} \right\rbrack \end{matrix}$

Meanwhile, the dehazed image J(x) must satisfy 0≦J(x)≦I(x). Thus, a range of the transmission rate t(x) is determined as Equation 5.

In Equation 5, the lower bound of transmission rate t_(LB)(x)=1−I(x)/A means transmission rate in the case where an object is invisible due to the heavy haze or the object has no radiance.

$\begin{matrix} {{1 - \frac{I(x)}{A}} \leq {t(x)} \leq 1} & \left\lbrack {{Equation}\mspace{14mu} 5} \right\rbrack \end{matrix}$

In the case where the same scene (the same distance between the camera and a point in a space) is taken under two different weather conditions (with different β values), the images of the scene taken by the camera can be described in Equation 6, respectively.

I ₁(x)=J(x)t ₁(x)+A ₁(1−t ₁(x))

I ₂(x)=J(x)t ₂(x)+A ₂(1−t ₂(x))  [Equation 6]

Further, the interrelation between transmission rates under two situations is described in the following Equation 7 and Equation 8.

$\begin{matrix} {\frac{{Int}_{1}(x)}{{Int}_{2}(x)} = {\frac{\beta_{1}}{\beta_{2}} = {P\mspace{14mu} ({Constant})}}} & \left\lbrack {{Equation}\mspace{14mu} 7} \right\rbrack \\ {{t_{1}(x)} = \left\lbrack {t_{2}(x)} \right\rbrack^{P}} & \left\lbrack {{Equation}\mspace{14mu} 8} \right\rbrack \end{matrix}$

Consequently, if the scattering coefficient rate under two situations and the transmission rate under one situation are known, the transmission rate under the other weather condition can be calculated. If the atmospheric brightness value is the same (i.e. A1=A2=A) and the transmission rate under one weather condition is the same as the lower bound of transmission rate t_(LB)(x), the transmission rate under a specific weather condition can be obtained from Equation 8, as Equation 9.

$\begin{matrix} {{{t(x)} = {\left\lbrack {1 - \frac{I(x)}{A}} \right\rbrack^{P} = \left\lbrack {t_{LB}(x)} \right\rbrack^{P}}},{0 \leq P < 1}} & \left\lbrack {{Equation}\mspace{14mu} 9} \right\rbrack \end{matrix}$

Herein, the scattering coefficient rate constant P is smaller than 1 because a scattering coefficient in the lower bound of transmission rate situation is bigger than that in other weather conditions.

FIG. 1 illustrates a flowchart of a method for image dehazing according to the present invention. In a step of image reading 101, a frame of an image taken by the camera is stored. If RGB color signals are input, both the input signal I^(c)(x) and the output signal J^(c)(x) are RGB color signals (i.e. cε{R, G, B}); if brightness signals are input, both the input signal I^(c)(x) and the output signal J^(c)(x) are brightness signals.

A step of atmospheric brightness value calculation 102 is a process of obtaining the atmospheric brightness value corresponding to a distance farthest away from the camera. The atmospheric brightness value is obtained as follows. As shown in Equation 10, a minimum value of RGB color channel for each pixel is calculated, and the atmospheric brightness value A^(c) is obtained by making a mean value of RGB values of pixels, which have the minimum value obtained by Equation 10 larger than a predetermined threshold value, of pixels on a current image. FIG. 2(G) illustrates an example where positions of pixels that have a minimum value obtained from the hazy input image by Equation 10 brighter than the top 1% of atmospheric brightness are shown as red.

min I(x)=min_(cε{R,G,B})(I ^(c)(x))  [Equation 10]

A step of transmission rate lower bound calculation 103 performs Equation 11 by receiving the hazy input signal I^(c)(x) and the atmospheric brightness value A^(c) output from the atmospheric brightness value calculation 102.

FIG. 2(B) illustrates an example of the lower bound of transmission rate obtained by Equation 11, wherein a heavily hazy area far away from the camera has a small value and a light hazy area close to the camera has a large value.

$\begin{matrix} {{t_{LB}(x)} = {1 - {\min_{\; {c\; \in {\{{R,G,B}\}}}}\left( \frac{I^{c}(x)}{A^{c}} \right)}}} & \left\lbrack {{Equation}\mspace{14mu} 11} \right\rbrack \end{matrix}$

A step of exponentiation operation 104 outputs [t_(LB)(x)]^(P) by multiplying the lower bound of transmission rate P times by Equation 9, wherein [t_(LB)(x)]^(P) corresponds to the initial transmission rate.

FIG. 2(C) illustrates an example of a result by multiplying the lower bound of transmission rate 0.75 times, wherein the transmission rate as a result of the above operation has a value larger than the lower bound of transmission rate for FIG. 2(B).

The initial transmission rate, as an output from the step of exponentiation operation 104, includes information on haze components included in an input image and reflection components unique to an object, whereby in the case of dehazing using the initial transmission rate, dehazing effect may be reduced since transmission rate for the reflection components is included.

A step of transmission rate correction 105 is a process of extracting transmission rate for haze components from the initial transmission rate. The haze components have a constant value locally; on the contrary, the reflection components vary according to changes in brightness (colors) included in the image. Accordingly, the transmission rate for the haze components included in the initial transmission rate corresponds to low-frequency components, and the transmission rate for the reflection components corresponds to high-frequency components.

Thus, the transmission rate for the haze components can be obtained by low-pass filter processing. However, when the transmission rate obtained by low-pass filter processing is directly applied to dehazing, dehazing performance is good but a halo effect occurs near an edge where changes in brightness are large. Accordingly, in the transmission rate correction process, by smoothing the initial transmission rate while having a strong edge relative to the initial transmission rate, the final transmission rate t(x) is calculated.

FIG. 2(D) illustrates an example of the low-frequency components of the initial transmission rate; FIG. 2(E) illustrates an example of the high-frequency components of the initial transmission rate; and FIG. 2(F) illustrates an example of the final transmission rate as a result of the transmission rate correction 105. As a result of the transmission rate correction from these examples, the heavily hazy part is similar to the low-frequency components of the initial transmission rate for FIG. 2(D), and the light hazy part having a sharp edge (an area of a person, and an area of a nearby leaf) is similar to the initial transmission rate for FIG. 2 (C).

A step of restored image calculation 106 outputs the dehazed image by performing Equation 12 with the input image I^(c)(x) and the final transmission rate t(x), and outputs a final restored image by improving brightness contrast of the dehazed image in the postprocessing 107.

$\begin{matrix} {{J^{c}(x)} = {\frac{{I^{c}(x)} - A^{c}}{\max \mspace{11mu} \left( {{t(x)},t_{0}} \right)} + A^{c}}} & \left\lbrack {{Equation}\mspace{14mu} 12} \right\rbrack \end{matrix}$

Herein, t₀ refers to a minimum transmission rate for preventing noise amplification caused when the transmission rate is very small. FIGS. 2(H) to 2(J) illustrate the output images restored by Equation 12. FIG. 2(H) illustrates a restored image, to which the initial transmission rate obtained by Equation 9 is applied; FIG. 2(I) illustrates a restored image, to which the initial transmission rate low-frequency components are applied; and FIG. 2(J) illustrates a restored image, to which a result of the transmission rate correction is applied. The restored image of FIG. 2(H), to which the initial transmission rate for FIG. 2(C) is applied, has no halo effect, but it has a low dehazing effect. The restored image of FIG. 2(I), to which the low-frequency components of the initial transmission rate for FIG. 2(D) are applied, shows a good dehazing performance, but a halo effect occurs at a location close to the camera. The restored image of FIG. 2(G), to which the final transmission rate as a result of correction of FIG. 2(F) is applied, shows a good dehazing performance without a halo effect.

In the case of a video clip, by determining whether there is an image to be processed, repetition is required, steps from image reading 101 to repetition 107 are repeated, or repetition is not required, the process is ended.

FIG. 3 illustrates an overall block diagram of the system for image dehazing by modifying a lower bound of a transmission rate and a method therefor of FIG. 1. Hereinafter, the operational process will be described. The atmospheric brightness value A^(c) corresponding to a distance farthest away from a camera relative to an input signal I^(c)(x) is calculated in the atmospheric brightness value calculation unit 300.

The lower bound of transmission rate t_(LB)(x) is calculated in the transmission rate lower bound calculation unit 301 by Equation 11. The value [t_(LB)(x)]^(P) obtained by multiplying the lower bound of transmission rate P times is output from the exponentiation operation unit 302, wherein the value [t_(LB)(x)]^(P) is the initial transmission rate.

The transmission rate correction unit 303 calculates the final transmission rate by correcting the initial transmission rate such that an output restored image is dehazed without a halo effect; the restored image calculation unit 304 calculates a dehazed image through operation of Equation 12; and the postprocessing unit 305 outputs the final restored image J^(c)(x) by increasing brightness contrast.

FIG. 4 illustrates an exemplary block diagram of transmission rate calculation method 1 of the transmission rate calculation block of FIG. 3; and FIG. 5 illustrates a dehazing process according to the transmission rate calculation method 1 of FIG. 4.

As a method for transmission rate correction to the initial transmission rate [t_(LB)(x)]^(P) output from an exponentiation operation unit 402, the final transmission rate is obtained by applying an edge reserved low-pass filter 403 to the initial transmission rate. An example of the edge reserved low-pass filter is a bilateral low-pass filter, wherein the edge reserved low-pass filter is configured such that weak low-pass filtering is performed at an edge and strong low-pass filtering is performed at a smooth area, thereby extracting low-frequency components while serving edge.

FIG. 5(A) illustrates an input image; and FIG. 5(B) illustrates a dehazed output image, which shows a good dehazing performance without a halo effect. FIG. 5(C) illustrates the initial transmission rate output from the exponentiation operation unit 402; and FIG. 5(D) illustrates the final transmission rate, to which 15×15 bilateral Gaussian low-pass filter, as the edge reserved low-pass filter 403, is applied, wherein compared with the initial transmission rate for FIG. 5(C), a part having a strong edge (an edge in light haze) has the initial transmission rate similar to that of FIG. 5(C), and on the contrary, strong low-pass filtering is performed at a part having weak edge.

FIG. 6 illustrates an exemplary block diagram of transmission rate calculation method 2; and FIG. 7 illustrates images dehazed by the transmission rate calculation method 2 of FIG. 6.

The transmission rate calculation method 2 is configured such that in order to correct the initial transmission rate [t_(LB)(x)]^(P) output from an exponentiation operation unit 402, the initial transmission rate is separated into low-frequency components t_(L)(x) and high-frequency components t_(H)(x) in an initial transmission rate separation unit 603.

FIG. 7(A) illustrates the initial transmission rate; and FIGS. 7(B) and 7(C) respectively illustrate the low-frequency components t_(L)(x) and the high-frequency components t_(H)(x) of the initial transmission rate separated by using a 15×15 average filter. The high-frequency components shown in FIG. 7(C) has a large value in the leaf in the light haze (close to the camera), and has a small value in a heavily hazy area (far away from the camera).

Large high-frequency components occurring in the strong edge should be compensated by the low-frequency components to prevent a halo effect; and small high-frequency components should be removed to improve definition of the restored image. The present invention is configured such that a weighting close to 1 is given to a large value of the high-frequency components, and a weighting close to 0 is given to a small value of the high-frequency components.

To reduce noise component from the high-frequency components t_(H)(x), the low-pass filter 604 is applied. As shown in Equation 13, a compression coefficient computing unit 605 calculates a compression coefficient value ω(x) proportional to an absolute value of the high-frequency components t_(H)(x) in consideration of the high-frequency components t_(H)(x) that have a large value in the area having a strong edge and have a small value in the smooth area, wherein the compression coefficient value ω(x) has a value close to 1 relative to the initial transmission rate corresponding to the strong edge, and has a value close to 0 relative to the smooth area.

The final transmission rate t(x) is obtained by adding a value t_(h)(x) obtained by multiplying the compression coefficient value ω(x) by the high-frequency components t_(H)(x) and the low-frequency components t_(L)(x) of the initial transmission rate.

$\begin{matrix} \begin{matrix} {{t_{h}(x)} = {{w(x)} \times {t_{H}(x)}}} \\ {= {{\min \left( {\frac{{F\left( {t_{H}(x)} \right)}}{D},1} \right)} \times {t_{H}(x)}}} \end{matrix} & \left\lbrack {{Equation}\mspace{14mu} 13} \right\rbrack \end{matrix}$

Herein, F(•) is an operator for low-pass filtering. FIG. 7(D) illustrates an example of the compression coefficient ω(x) having a value close to 1 in the leaf in the light haze (close to the camera), and having a value close to 0 in a heavily hazy area. FIG. 7(E) illustrates an example of the compressed high-frequency components t_(h)(x); and FIG. 7(F) illustrates the final transmission rate. FIGS. 7(G) and 7(H) illustrate an input image and a dehazed restored image.

FIG. 8 illustrates an exemplary block diagram of transmission rate calculation method 3; and FIG. 9 illustrates images dehazed by the transmission rate calculation method 3 of FIG. 8.

The initial transmission rate [t_(LB)(x)]P output from an exponentiation operation unit 802 is separated into low-frequency components t_(L)(x) and high-frequency components t_(H)(x) in an initial transmission rate separation unit 803.

A threshold value processing unit 804 expresses a position of a component of the high-frequency components t_(H)(x) of the initial transmission rate greater than a threshold value as 1.

An independent component removal unit 805 removes positional components showing a large edge caused by noise in the output process from the threshold value processing unit 804.

An extension processing unit 806 performs an extension processing on a component expressed as 1 output from the independent component removal unit so as to output a value R(x) having a position of an extended area expressed as 1 and a position of a rest expressed as 0.

Accordingly, a value t_(h)(x) output from a multiplication operator 807 is as follows: a value R(x) having 1 output from the extension processing unit is the same position as the initial transmission rate t_(H)(x), and a position of a rest has 0.

An addition operator 808 outputs the final transmission rate t(x) by adding the modified high-frequency components t_(h)(x) of the initial transmission rate and the low-frequency components t_(L)(x).

FIG. 9(A) illustrates the initial transmission rate; and FIGS. 9(B) and 9(C) illustrate the low-frequency components t_(L)(x) and the high-frequency components t_(H)(x) of the initial transmission rate. FIG. 9(D) illustrates a process of calculating a value R(x) in the extension processing unit 806, wherein a red color shows a position of a component having a value larger than a threshold value by the threshold value processing unit 804, and a yellow color shows an extension area by extension processing unit. A position of the red not included in the yellow part shows positional components removed by the independent component removal unit 805. FIG. 9(E) illustrates the modified high-frequency components t_(h)(x); and FIG. 9(F) illustrates the final transmission rate. FIGS. 9(G) and 9(H) illustrate an input image and a dehazed restored image.

The present invention provides a clear image by removing components that lower visibility, from an image having lowered visibility due to mixture of light and color components of an object and light and color components, such as haze, fog, clouds, and the like, in the air. In particular, the present invention requires a small amount of calculating, which enables real-time processing in an embedded processor, and has a good dehazing performance. Thus, in the case where the present invention is applied to a high resolution monitoring system, an image black box for automobile, fire prevention system, and the like, lowered visibility problems due to fog or smoke can be solved. Further, the present invention can be applied to advanced safety vehicles that are recently under study. Further, the present invention can be applied to various smartphone apps relating to images since it is possible to process high-definition image in mobile phones. Currently, color coordinate systems used in most multimedia systems use a brightness signal and a chrominance signal like YCbCr color coordinate system, rather than an RGB coordinate system. Thus, the present invention has an excellent performance with only brightness information, whereby it is easy to be adapted to multimedia system without conversion processing of a color coordinate system.

Description of reference characters used in the embodiment is as follows.

I(x): brightness value of input pixel in x coordinate

J(x): brightness value of dehazed output pixel in x coordinate

t(x): transmission rate in x coordinate (as a value expressing a degree of haziness, 0 referring to opacity, and 1 referring to transparency)

A: atmospheric brightness value of a pixel corresponding to a distance farthest away from camera

t_(LB)(x): lower bound of transmission rate for input pixel in x coordinate

t_(L)(x): transmission rate (low-frequency components) for haze components of transmission rate in x coordinate

t_(H)(x): transmission rate (high-frequency components) for reflection components of transmission rate in x coordinate

sel_comp(•): selective compression operation

t_(h)(x): transmission rate (high-frequency components) for reflection components in x coordinate processed by selective compression

cε{R, G, B}: R, G, B color channel

I^(c)(x): brightness value of input color channel c in x coordinate

J^(c)(x): brightness value of output color channel c in x coordinate

Min_(c)ε_({R, G, B})(I^(c)(x)): operation for obtaining minimum value of RGB color channel in x coordinate

[a(x)]^(P): signal a in an x coordinate raised to Pth

D: threshold value used for selective compression

t₀: minimum transmission rate

Although the present invention has been described in conjunction with the preferred embodiments which illustrate the technical spirit of the present invention, it will be apparent to those skilled in the art that the present invention is not limited only to the illustrated and described configurations and operations themselves but variations and modifications are possible without departing from the scope of the spirit of the invention. Accordingly, all of appropriate variations, modifications and equivalents are considered to pertain to the scope of the present invention. 

What is claimed is:
 1. A system for image dehazing by modifying a lower bound of a transmission rate, the system comprising: an atmospheric brightness value calculation unit for calculating an atmospheric brightness value (A^(c)) corresponding to a distance farthest away from a camera relative to an input signal [I^(c)(x)]; a transmission rate lower bound calculation unit for calculating a lower bound of a transmission rate [t_(LB)(x)] by receiving the input signal and the atmospheric brightness value from the atmospheric brightness value calculation unit; an exponentiation operation unit for operating an initial transmission rate {[t_(LB)(x)]^(P)} by multiplying the lower bound of transmission rate P times; a transmission rate correction unit for calculating a final transmission rate by correcting the initial transmission rate such that an output restored image is dehazed without a halo effect; a restored image calculation unit for calculating a dehazed image through operation; and a postprocessing unit for outputting a final restored image [J^(c)(x)] by increasing brightness contrast.
 2. The image dehazing system of claim 1, wherein the transmission rate lower bound calculation unit calculates the lower bound of transmission rate [t_(LB)(x)] by Equation
 11. $\begin{matrix} {{t_{LB}(x)} = {1 - {\min_{\; {c\; \in {\{{R,G,B}\}}}}\left( \frac{I^{c}(x)}{A^{c}} \right)}}} & \left\lbrack {{Equation}\mspace{14mu} 11} \right\rbrack \end{matrix}$
 3. The image dehazing system of claim 1, wherein the restored image calculation unit calculates the dehazed image by Equation
 12. $\begin{matrix} {{J^{c}(x)} = {\frac{{I^{c}(x)} - A^{c}}{\max \mspace{11mu} \left( {{t(x)},t_{0}} \right)} + A^{c}}} & \left\lbrack {{Equation}\mspace{14mu} 12} \right\rbrack \end{matrix}$
 4. The image dehazing system of claim 1, wherein the final transmission rate is obtained by applying an edge reserved low-pass filter to the initial transmission rate.
 5. The image dehazing system of claim 1 further comprising: an initial transmission rate separation unit arranged to separate the initial transmission rate into low-frequency components [t_(L)(x)] and high-frequency components [t_(H)(x)]; and a compression coefficient computing unit arranged such that the high-frequency components are low-pass filtered to obtain a compression coefficient value [ω(x)] proportional to an absolute value of the initial transmission rate having been low-pass filtered as shown in Equation 13, and the final transmission rate [t(x)] is obtained by adding a value [t_(h)(x)] obtained by multiplying the compression coefficient value by the high-frequency components and the low-frequency components of the initial transmission rate. $\begin{matrix} \begin{matrix} {{t_{h}(x)} = {{w(x)} \times {t_{H}(x)}}} \\ {= {{\min \left( {\frac{{F\left( {t_{H}(x)} \right)}}{D},1} \right)} \times {t_{H}(x)}}} \end{matrix} & \left\lbrack {{Equation}\mspace{14mu} 13} \right\rbrack \end{matrix}$
 6. The image dehazing system of claim 1 further comprising: an initial transmission rate separation unit arranged to separate the initial transmission rate into low-frequency components [t_(L)(x)] and high-frequency components [t_(H)(x)]; a threshold value processing unit for expressing a position of a component of the high-frequency components [t_(H)(x)] of the initial transmission rate greater than a threshold value as 1; an independent component removal unit for converting independently existing 1 output from the threshold value processing unit into 0; an extension processing unit configured to perform an extension processing on a component expressed as 1 output from the independent component removal unit so as to output a value [R(x)] having a position of an extended area expressed as 1 and a position of a rest expressed as 0; a multiplication operator for obtaining modified high-frequency components [t_(h)(x)] by multiplying the value [R(x)] by the high-frequency components of the initial transmission rate; and an addition operator for outputting the final transmission rate [t(x)] by adding the modified high-frequency components [t_(h)(x)] of the initial transmission rate and the low-frequency components [t_(L)(x)].
 7. A method for image dehazing by modifying a lower bound of a transmission rate, wherein the method uses a system for image dehazing by modifying a lower bound of a transmission rate, the image dehazing system including: an atmospheric brightness value calculation unit, a transmission rate lower bound calculation unit, an exponentiation operation unit, a transmission rate correction unit, a restored image calculation unit, and a postprocessing unit, the method comprising: calculating a brightness value relative to an atmospheric brightness value of an input image by using the atmospheric brightness value calculation unit; calculating a lower bound of a transmission rate for each pixel using information on the input image and the atmospheric brightness value by using the transmission rate lower bound calculation unit; obtaining an initial transmission rate by an exponentiation operation with the lower bound of transmission rate by using the exponentiation operation unit; correcting the input image to have a strong edge relative to the initial transmission rate by smoothing such that an output image restored by the transmission rate correction unit is provided to be a dehazed clear image without halo effect; calculating a final transmission rate to be used to dehaze by using the restored image calculation unit; and outputting a dehazed restored image processed by image restoration and postprocessing by using the postprocessing unit. 